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Abstract: Purpose: The null-hypothesis that retention loss of resin fissure sealants predicts caries manifestation no more 
accurately than random values was tested. 

Methods: Systematic reviews were checked and electronic databases searched for clinical trials. Trials reporting on the re- 
tention of resin sealants and caries occurrence in permanent molar teeth, with minimum 24-month follow-up period, were 
included. Extracted data: number of sealed teeth, number of teeth without completely retained sealants, number of sealed 
teeth with caries. The number of teeth with complete sealant retention and absence of carious lesions/cavities was calcu- 
lated; the predictive outcomes: true/false positive; false/true negative were established. Random values were generated as 
control-data. Diagnostic Odds ratios (DOR) were computed and tested for statistical difference. Summary Receiver Oper- 
ating Characteristic curves were plotted. 

Results: 95 trials were found. Median DOR values were 1.21 and 0.28 for test- and control data, respectively. Wilcoxon 
test (z = 0.56; p = 0.58) and Sign test (z = 1.38; p = 0.17) results were statistically non-significant. The null-hypothesis 
was not rejected. 

Conclusions: Predictions based on the retention loss of resin sealants, regarding caries manifestation, was no more accu- 
rate than random guesses. Sealant retention loss appears not to be a valid predictor for clinical outcome. 

Keywords: Diagnostic Odds ratio, fissure sealant, resin, retention, surrogate endpoint, tooth caries. 



INTRODUCTION 

Most carious lesions and cavities develop in pits and fis- 
sures of occlusal surfaces in primary and permanent poste- 
rior teeth [1]. The simultaneous interaction of a susceptible 
host, cariogenic microflora and suitable substrate (i.e. fer- 
mentable carbohydrates from food in the oral environment) 
has been established as the etiological factor [2]. Exclusion 
of the cariogenic microflora from substrate may prevent 
carious lesion and cavity development and is achieved by 
placing a physical barrier in the form of a seal on pits and 
fissures [3]. 

Pit and fissure sealants have evolved, from phosphate 
cements [4], silver nitrate [5] and black copper cement [6], to 
the current resin- based materials [7]. Buonocore (1955) re- 
ported the first clinical resin-based fissure sealant experi- 
ences [8]. Subsequent clinical research compared the devel- 
opment of carious lesions/cavities in sealed teeth with that in 
non-sealed teeth as controls, mainly in split mouth trials, and 
established retention longevity of the sealant material func- 
tions as a beneficial factor in the prevention of caries [9-11]. 
A subsequent systematic review by the Cochrane collabora- 
tion, confirmed these findings [12]. 
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On the basis of the positive results, the further use of a 
control group of non-sealed teeth in clinical studies became 
ethically unacceptable [13]. Instead, subsequent studies 
compared newer with older sealant materials and/or sealant 
techniques as their controls [14]. The outcome measure of 
fissure sealant studies also changed, from the rate of caries 
manifestation to the retention rate of fissure sealants. Based 
on the observation of 'retention' as a beneficial factor in the 
prevention of caries, the rate of sealant retention was forth- 
with assumed to be a valid surrogate endpoint for fissure 
sealant effectiveness. Rock and Anderson (1982) highlighted 
it as obvious that (resin-based) fissure sealants are effective 
only because of their presence on the teeth. Regression re- 
sults of retention rate with caries manifestation supported 
this logical conjecture [15]. Weintraub and Bart (1987) 
agreed that the choice of monitoring the retention rate in 
fissure sealant studies as the expression of effectiveness was 
based on the fact that sealant effectiveness was a direct func- 
tion of its retention [16]. Liebenberg (1994) found it axio- 
matic that if any prophylactic benefit is to be accrued, reten- 
tion of the sealant must be completely intact [17]. Locker et 
al. (2003) and Muller-Bolla et al. (2006) stated that since 
caries does not develop as long as the sealant remains adher- 
ent to the tooth, the length of the time such sealant is re- 
tained is justified for use as a surrogate measure of its effec- 
tiveness in preventing tooth decay [13, 14]. Beauchamp et al. 
(2008) declared, on behalf of the American Dental Associa- 
tion Council on Scientific Affairs, that since the effective- 
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ness of sealants depends on clinical retention, the retention 
rate is acceptable as proxy for caries prevention [18]. Sasa 
and Donly (2010) also accepted that sealant retention equates 
with caries prevention [19] and Kiihnisch et al. (2012) de- 
clared that because optimum protection is only guaranteed if 
the sealant completely covers all pits and fissures, 'intact 
sealant' (as opposed to lost or partially retained sealant) is 
the leading fissure sealant criterion today [20]. 

For the purpose of this study and in accordance with the 
general consensus [15-20], 'complete retention' was as- 
sumed to be a valid surrogate endpoint for caries prevention. 
On the basis of the further consensus that sufficiently correct 
prediction of a true clinical endpoint by its surrogate end- 
point is an essential requirement for surrogate validity [21 - 
25], it was also assumed that 'complete retention', as the 
valid surrogate endpoint, would successfully predict caries 
manifestation. In this context, the aim of this study was to 
test the null-hypothesis (H 0 ) that the rate of losing complete 
retention of resin- based fissure sealants, after an observation 
period of at least 24-months, does not predicts caries mani- 
festation on pits and fissures of permanent molar teeth sig- 
nificantly more accurately than any random values (with the 
alternate hypothesis, H b that complete retention predicts 
caries manifestation more accurately). 

MATERIALS AND METHODOLOGY 

Literature Search 

The references and additional online content of three sys- 
tematic reviews [20, 26, 27] on the topic of fissure sealants 
were checked for relevant trials. PubMed/Medline, Directory 
of Open Access Journals (DOAJ); IndMed and Scielo were 
searched by both authors (SM and VY), independently, using 
the search term: "fiss* AND seal*" in order to identify addi- 
tional studies. The search period was limited to the publica- 
tion period for the electronic database search from 
30.09.2011 to 21.10.2012. 

Titles and abstracts of the resulting citations were 
scanned for possible inclusion in line with the following in- 
clusion criteria: 

1. Clinical study reporting on the retention and caries 
occurrence of resin-based fissure sealants in perma- 
nent molar teeth (no distinction was made between 
different types of resin-based sealants); 

2. Minimum 24-month follow-up period; 

Articles whose titles and abstracts were in alignment with 
the inclusion criteria were retrieved in full copy and were 
reviewed by both authors of this article. Articles were further 
excluded if: premolars were included in the study and their 
data analysed together with that of molar teeth; no caries was 
assessed; no computable data was reported; the number of 
teeth evaluated for retention and caries was not the same; the 
number of previously sealed teeth that developed carious 
lesions/cavities was reported together with the number of 
teeth filled and extracted. Articles that could not be traced in 
full copy were also excluded. Disagreements were resolved 
through discussion and consensus. 
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Data Extraction and Statistical Analysis 

Both authors extracted data from the accepted articles in- 
dependently without being blinded to authors, institutions, 
journal names and trial results. The extracted data included: 
the number of evaluated sealed teeth (N); the number of 
teeth without completely retained fissure sealants (n R _) - loss 
of complete material retention; the number of sealed teeth 
with carious lesion/cavities (n c .) at the end of each follow-up 
period. The choice of using loss of 'complete material reten- 
tion' was based on the consideration that "intact sealant"; i.e. 
complete sealant without material loss, has been adopted as 
the leading fissure sealant criterion today [20]. From the ex- 
tracted n R _ and n c . -values the number of teeth with complete 
sealant retention (n R+ ) and absence of carious lesions/cavities 
(n c +) were calculated (n - N). Based on the n R+ ; n R _; n c _ and 
n c + study data, the following predictive outcomes were es- 
tablished: 

1. True positive outcome (TP): The number of n R _ - teeth 
where carious lesion/cavities (n c .) have developed; 

2. False positive outcome (FP): The number of n R _ - teeth 
where no carious lesion/cavities (n c +) have developed; 

3. False negative outcome (FN): The number of n R+ - 
teeth where carious lesion/cavities (n c _) have devel- 
oped; 

4. True negative outcome (TN): The number of n R+ - 
teeth where no carious lesion/cavities (n c +) have de- 
veloped. 

The predictive outcomes based on the retention rates ex- 
tracted from fissure sealants studies were considered as the 
test-data. In order to establish whether retention loss of resin 
sealants can predict caries manifestation more accurately 
than any random guesses by play of chance, random values 
were generated as control-data by using an online random 
number generator [28]. One random number between 0 and 
(N) was generated for each study dataset and these were used 
instead of the n R _ values of the test data. Based on these ran- 
dom numbers (n rand ), predictive outcome values were calcu- 
lated and formed the control data. 

From the predictive test- and control outcomes, Diagnos- 
tic Odds Ratios (DOR) were computed per study dataset. 
The DOR combines sensitivity and specificity characteristics 
into one single predictive summary measure and is defined 
as: DOR = (TP x TN) / (FP x FN) [29]. The DOR may range 
from zero to infinity. DOR values of less than 1 .00 indicate 
that the prediction may be useful, by inverting its outcome. 
A DOR value of or close to 1 .00 provides no predictive evi- 
dence and corresponds to the rising diagonal in Summary 
Receiver Operating Characteristic (SROC) graphs. The 
higher the DOR value (>1.00) the better the predictive accu- 
racy [30]. 

In addition to the computation of DOR values, SROC 
curves with a 95% Confidence interval (CI) were plotted for 
the test- and control data. SROC curves in relation to the 
rising diagonal of the SROC graph (DOR = 1.00) give a 
graphical representation of the accuracy of predictions ac- 
cording to the established true positive (sensitivity, shown on 
the y- axis) and false positive rates (1 -specificity, shown on 
the x- axis). 



104 The Open Dentistry Journal, 2013, Volume 7 



Mickenautsch and Yengopal 



Statistical software Meta-DiSc Version 1.4 [31] was used 
for computing and plotting all DOR values and SROC curves. 
Zero cells were converted into 0.5 values prior to computation. 
The median values with a 25-75% percentile range of the 
DOR values were computed for the test- and control data and 
the results plotted in a Box-Whisker graph. Meta-analytic 
pooling of the individual DOR values was only considered if a 
low in-between-study-heterogeneity was detected (I 2 < 30%) 
[32]. Because the samples of DOR values from both test and 
control data shared the same caries measurements, they were 
considered as dependent data samples [33]. Therefore, median 
differences between both samples were tested for statistical 
significance, using the Wilcoxon-Matched-Pair test and Sign 
test (Biostat 2009 statistical software). Alpha level for statisti- 
cal significance was set at 5%. Rejection of the null- 
hypothesis was conditional on the basis of the median DOR of 
the test data being statistically significantly larger than that of 
the control data (p < 0.05). 

Sensitivity Analysis 

In order to test whether multiple measurements of the 
same materials at different observation periods may have had 
any influence on the results, sensitivity analysis was con- 
ducted by retaining only datasets of the longest observation 
period per sealant material. The following 16 datasets were 
subsequently removed: DS 10, 12, 27, 28, 31, 32, 64, 66, 80, 
104, 118, 120, 122, 162-164 and the remaining data samples 
were re-analysed. 

RESULTS 

Literature Search and Data Selection 

The literature search generated 95 citations, of which 90 
were extracted from the additional online content and refer- 
ence lists of three systematic reviews [20, 26, 27] and 5 cita- 
tions were identified through an electronic database search. 
Details of study design and length of observation period per 
study are shown in APPENDIX FILE 1. From the 95 stud- 
ies, 185 datasets (DS) with relevance to sealant retention and 
caries development were extracted. From the 185 datasets, 
95 (from 57 studies) were excluded for the following rea- 
sons: exclusive investigation of molar teeth not made explicit 
(10 DS); caries assessment missing (28 DS); full article 
could not be traced (21 DS); no computable data reported (3 
DS); not the same number of teeth evaluated for retention 
and caries (9 DS); number of previously sealed teeth that 
developed carious lesions/cavities reported together with 
number of teeth filled and extracted (24 DS). 

Extracted Data and Statistical Analysis 

From the provisionally included 185 datasets, a total of 
90 datasets were accepted for data analysis (APPENDIX 
FILE 2). For the purpose of this study no distinction was 
made between tooth- or side-specific data. High statistical 
in-between-study heterogeneity was established (I 2 = 95.4% 
and 96.4% for test- and control data, respectively). Thus, no 
meta-analytic pooling of DOR values was considered. 

The computed DOR values per dataset for both data 
groups are shown in APPENDIX FILE 3. Summary Re- 
ceiver Operating Characteristic (SROC) curves for the test 



and control data are shown in Figs. (1 and 2). The median 
DOR values were 1.21 (0.20 - 10.71) and 0.28 (0.07 - 
13.10) for the test and control data, respectively (Fig. 3). The 
results of the Wilcoxon test (z = 0.56; p = 0.58) and Sign test 
(z = 1.38; p = 0.17), which evaluated the differences between 
medians for DOR values, were statistically non-significant. 
The null-hypothesis that the rate of losing complete retention 
of resin- based fissure sealants, after an observation period of 
at least 24-months, does not predict caries manifestation on 
pits and fissures of permanent molar teeth significantly more 
accurately than any random values could therefore not be 
rejected. 

Sensitivity Analysis 

Sensitivity analysis results showed median DOR values 
of 0.98 (0.20 - 6.35) and 0.20 (0.06 - 13.10) for the test- and 
control data, respectively. The results of the Wilcoxon test (z 
= 0.47; p = 0.64) and Sign test (z = 1.30; p = 0.20) were sta- 
tistically non-significant. Multiple measurements of the same 
materials at different observation period points appear to 
have had no decisive influence on the results and the conclu- 
sions of this study. 

DISCUSSION 

Limitations of Study Method 

The aim of this study was to test the null-hypothesis that 
the rate of losing complete retention of resin based fissure 
sealants, after an observation period of at least 24-months, 
does not predict caries manifestation on pits and fissures of 
permanent molar teeth significantly more accurately than any 
random values. 

In order to meet this aim, the chosen study methodology 
focused on the data extracted from resin based fissure sealant 
trials, without taking aspects of internal trial validity under 
consideration. Such aspects, particularly related to attrition- 
and detection bias, may have affected the trial results and thus 
in turn, the data on which this study is based. Neither, how- 
ever, was internal trial validity with specific focus on system- 
atic errors/biases investigated during the systematic review by 
Kiihnisch et al, 2012 [20], whose references (provided in its 
additional online content) constituted the main data source of 
this investigation (APPENDIX FILE 1). Consequently, this 
study is in line with the same assumptions concerning internal 
validity of the same included fissure sealant trials as those 
reviewed by Kiihnisch et al, 2012 [20]. 

In this study, more than half of the available datasets 
were excluded (APPENDIX FILE 1). However, the exclu- 
sion of premolar teeth is in line with investigations into seal- 
ant effectiveness reported by other authors [20] and is based 
on the observation that caries develops less on premolar 
teeth than on molars [15]. The exclusion of 24 datasets on 
the basis of their inclusion of the filled and extracted DMFT 
component appears justified, as the true reasons for the fill- 
ing and extraction of formerly sealed teeth are unknown. 
Other reasons for data exclusion: lack of computable data 
reported, different numbers of teeth assessed for retention 
and caries and lack of caries assessment are clear indicators 
for non-relevance with regard to the study aim. The lack of 
21 datasets from trials that remained untraceable in full copy 
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Analysis Options: 

Add 1/2 to all cells of the studies with zero 
Filter OFF 

Symmetric SROC curve fitted using Moses' Model [Weighted regression (Inverse Variance) 
Defined relevant region:AII ROC spac 

AUC = Area under curve; SE = Standard error. 
Fig. (1). SROC curve for test data: retention rate. 
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AUC = Area under curve; SE = Standard error. 
Fig. (2). SROC curve for control data: random values. 
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Fig. (3). Median diagnostic Odds ratios (DOR). 

through the available library- and Internet sources or directly 
from the authors remains a concern. However, these would 
have comprised just 19% of the total available data and 
whether some of the missing data would not have been ex- 
cluded for other reasons too, if the trial reports could have 
had been traced, remains doubtful. Under such assumption, 
the lack of the missing data may not have significantly im- 
pacted the conclusion of this study. 

Study Results 

Treatment effects on surrogate endpoints should reliably 
predict the effect of such treatment on the true clinical end- 
point [21-25]. Hence, if the placement of resin-based seal- 
ants on pits and fissures of permanent molar teeth result in 
the retention/loss of the material after >24 months, this 
should reliably predict the absence/presence of caries mani- 
festation beyond the play of chance. Within the context of 
this study, the number of teeth with/without completely re- 
tained sealants (n R+ /n R .) should be able to predict reasonably 
the number of teeth that do not/do develop carious decay in 
pits and fissures (n c +/n c _). Such results form the true nega- 
tive/positive (TN/TP) predictions. In comparison, the num- 



ber of false negative/positive (FN/FP) results should not ex- 
ceed between 2.5 - 10% of all predictions [21]. 

In this study, the test data for sealant retention (n R _) 
was obtained from clinical fissure sealant trials and com- 
pared to random values (n rand ) in relation to the true clinical 
endpoint (= caries manifestation / n c .) and computed as me- 
dian Diagnostic Odds ratios (DOR). It was expected, on the 
basis of current consensus concerning fissure retention [15- 
20], that the predictive DOR values from the test data would 
be significantly higher (more accurate) than those from the 
control. However, no statistically significant differences be- 
tween the TP/TN and FP/FN dependent DOR values were 
found (Wilcoxon test: z = 0.56; p = 0.58; Sign test: z = 1.38; 
p = 0.17). The null-hypothesis (H 0 ) could therefore not be 
rejected, thus no sufficient evidence in support of the alter- 
nate hypothesis (Hi) was found. These results suggest that 
predictions based on the relationship of the retention rate of 
resin-based fissure sealants and caries manifestation are no 
more accurate than random guesses. Furthermore, the SROC 
curve (Fig. 2), based on such random values (median DOR 
0.28), appears to indicate a higher (albeit inverted and still 
purely random) predictive accuracy than the sealant retention 
rate (DOR = 1.21; Fig. 1). 
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The lack of any significant difference in predictive 
accuracy between the retention rate and random values may 
not be surprising. Many factors other than the mere loss of 
complete sealant retention have a potential effect on caries 
manifestation in pits and fissures of permanent molar teeth. 
These factors may include: the position of the molar tooth in 
the mouth, the period between retention loss and follow-up, 
access to fluoride, oral hygiene and dietary habits, enamel 
structure, saliva factors or, possibly, factors that remain un- 
known. Manifestation of caries may also be related to sealed 
pits and fissures, if the partially lost sealants were misclassi- 
fied as 'completely retained' or if pits and fissures still con- 
tain micro-remnants of sealant materials that provide caries 
protection but were classified as 'loss of complete retention'. 

Against this background, it may appear obvious that the 
retention rate alone is unable to account for the influence of 
other known and unknown factors that may influence caries 
manifestation. Nonetheless, the retention rate of fissure seal- 
ants has been accepted as valid surrogate for caries preven- 
tion [21-25]. Valid surrogate endpoints are defined as meas- 
urements or physical signs for use as substitutes for true 
clinical endpoints [34], while true clinical endpoints are de- 
fined as clinical, patient-relevant events of which the patient 
is aware, which the patient wants to avoid and which affect 
her/his quality of life [21]. Caries manifestation in pits and 
fissures constitutes a true clinical endpoint that was utilized 
during first generation of fissure sealant trials [9-11]. In light 
of the inability of the retention rate to be an accurate predic- 
tor of caries in pits and fissures, further (randomised con- 
trolled) sealant trials should have retained this clinical end- 
point as the outcomes measure while using resin-based seal- 
ant materials and sealant techniques, established during the 
previous trials, as gold standards against which new materi- 
als/techniques were to be compared. 

The use of invalid surrogates that cannot sufficiently pre- 
dict their clinical endpoints carries the danger of unexpected 
adverse effects in clinical trials, therapeutic uncertainties due 
to ambiguous evidence, as well as the risk of rejection of 
potentially useful therapies because they do not show bene- 
fits in line with the surrogate endpoints [24]. Of these conse- 
quences, the latter is especially apparent with regard to glass- 
ionomer cement (GlC)-based fissure sealants [20], despite 
the fact that systematic reviews have found no difference 
between the caries-preventive effect of GIC and that of resin- 
based sealants [26, 27]. For these reasons, it has been repeat- 
edly argued that: (i) biological outcomes should take prece- 
dence over mechanical ones; (ii) because sealants are placed 
in pits and fissures in order to prevent the onset of caries 
manifestation or to arrest it, the true outcome of such inter- 
vention should therefore be expressed in terms of how well 
such intervention has managed to achieve this objective [35, 
36]. 

CONCLUSIONS 

The retention rate of resin-based fissure sealants is not a 
valid predictor for clinical outcomes. Nevertheless, complete 
retention of sealants in pits and fissures has been established 
as a beneficial factor in preventing the development of cari- 
ous lesions/cavities. On this basis the aim of achieving com- 
plete retention, regardless of the type of material, remains an 



important clinical consideration in the placing of fissure 
sealants. However, inter- category misinterpretation of 'bene- 
ficial factors' as 'valid surrogate endpoints' should be 
avoided. In order for a surrogate endpoint to be valid, it 
needs not only to be a beneficial factor but also to be a suffi- 
ciently accurate predictor (beyond the play of chance) of its 
true clinical endpoint. The retention rate of fissure sealants 
appears not to fulfil this criterion and thus seems unable to 
replace 'caries manifestation' as the appropriate study out- 
come in clinical fissure sealant trials. 
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